how to copy s3 object from one bucket to another using python boto3

Question

I want to copy a file from one s3 bucket to another. I get the following error:

s3.meta.client.copy(source,dest)
TypeError: copy() takes at least 4 arguments (3 given)

I'am unable to find a solution by reading the docs. Here is my code:

#!/usr/bin/env python
import boto3
s3 = boto3.resource('s3')
source= { 'Bucket' : 'bucketname1','Key':'objectname'}
dest ={ 'Bucket' : 'Bucketname2','Key':'backupfile'}
s3.meta.client.copy(source,dest)

Dishabille

This answer works https://mcmap.net/q/302915/-move-files-between-two-aws-s3-buckets-using-boto3 – Cachexia

Answer 1

You can try:

import boto3
s3 = boto3.resource('s3')
copy_source = {
      'Bucket': 'mybucket',
      'Key': 'mykey'
    }
bucket = s3.Bucket('otherbucket')
bucket.copy(copy_source, 'otherkey')

or

import boto3
s3 = boto3.resource('s3')
copy_source = {
    'Bucket': 'mybucket',
    'Key': 'mykey'
 }
s3.meta.client.copy(copy_source, 'otherbucket', 'otherkey')

Note the difference in the parameters

Difference

How does this change when the source bucket is a public s3 bucket not under my account? – Azalea

@Azalea Check this out: aws.amazon.com/premiumsupport/knowledge-center/… Hope it helps. – Difference

Is it just me or is this an extremely weird function? Source is a dictionary but destination is two strings? WTH AWS? And of course the key names are capitalized just to make absolute sure they don't abide by any convention... – Hawaiian

@Hawaiian Yup it is not the best way to go about it. Haven't checked the recent revision. Probably it will be fixed. If not can create an issue in their repo. – Difference

Answer 2

Since you are using s3 service resource, why not use its own copy method all the way?

#!/usr/bin/env python
import boto3
s3 = boto3.resource('s3')
source= { 'Bucket' : 'bucketname1', 'Key': 'objectname'}
dest = s3.Bucket('Bucketname2')
dest.copy(source, 'backupfile')

Charitacharitable

copy() missing 1 required positional argument: 'Key' – Webbing

Answer 3

This is the syntax from docs:

import boto3

s3 = boto3.resource('s3')
copy_source = {
    'Bucket': 'mybucket',
    'Key': 'mykey'
}
s3.meta.client.copy(copy_source, 'otherbucket', 'otherkey')

Reassure